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Abstract —  This  paper  explores  the  problem  of  cooperative 
control  among  multiple  networked  unmanned  air  vehicles 
(UAVs)  for  persistent  area  denial  (PAD)  mission.  An  adaptive 
Markov  chain  model  is  used  to  predict  the  locations  of  pop-up 
threats.  The  mixed  information  of  predicted  pop-up  threats 
and  actual  pop-up  targets  is  utilized  to  develop  cooperative 
strategies  for  networked  UAVs.  The  approach  is  illustrated 
by  use  of  a  simulation  test  bed  for  multiple  networked  UAVs 
and  Monte  Carlo  simulation  runs  to  evaluate  our  cooperative 
strategy.  Both  theoretical  analysis  and  simulation  results  are 
presented  to  demonstrate  the  effectiveness  of  using  predicted 
pop-up  information  in  improving  the  overall  PAD  mission 
performance. 


I.  Introduction 

ninhabited  air  vehicles  have  been  identified  as 
potential  valuable  resources  for  future  military, 
communications  and  earth- science  efforts. 

Cooperation  among  multiple  UAVs  is  a  key  capability  for 
utilizing  the  full  potential  of  UAV  systems.  With  respect  to 
military  applications,  one  of  the  potential  missions  for 
UAVs  is  persistent  area  denial.  The  aim  of  multiple  UAVs 
in  PAD  operations  is  to  provide  persistent  surveillance, 
tracking,  and  rapid  engagement  with  high-volume  strike, 
against  threats  (e.g.,  enemy  integrated  air  defense  system) 
at  various  ranges  in  the  adversarial  terrains.  Threats  in  the 
battle  space,  in  general,  are  of  two  types:  known  and  pop¬ 
up.  The  locations  of  known  threats  are  identified  by  the 
UAVs  at  the  beginning  of  the  battle,  while  the  locations  of 
pop-up  targets  are  not  always  observable  during  the  entire 
PAD  operation.  In  other  words,  the  pop-up  targets  may 
appear  and  disappear  at  frequent  and  random  intervals  of 
time.  Obviously,  uncertainty  introduced  by  pop-up  targets 
presents  the  significant  theoretical  and  technical  challenges 
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in  designing  and  implementing  cooperative  control 
strategies  for  multiple  UAVs. 

The  present  paper  is  mainly  concerned  with  the  problem 
of  coordinating  multiple  UAVs  for  pop-up  threats  in  the 
PAD  operation.  In  this  work,  the  most  closely  related  work 
is  introduced  in  [1]  and  [2].  In  [1],  the  author  modeled  the 
information  flow  and  communication  constraints  between 
UAVs  and  developed  cooperative  strategy  for  path  re¬ 
planning  to  deal  with  pop-up  threats  during  the  mission. 
The  work  in  [1],  however,  does  not  consider  that  the  pop¬ 
up  targets  appear  and  disappear  in  a  stochastic  manner.  In 
[2],  the  locations  of  pop-up  targets  are  modeled  as  a  first- 
order  Markov  chain.  Utilizing  a  mixed  integer 
programming  approach,  a  cooperative  control  strategy  of 
multiple  UAVs  for  PAD  operation  is  developed  in  [2]  as 
well.  In  this  strategy,  the  trajectories  of  vehicles  are 
optimized  so  that  they  can  reach  all  targets  in  the  shortest 
time.  The  predicted  information,  however,  is  not  utilized  in 
the  computation  of  cooperative  control  strategies.  In  this 
paper,  we  modify  this  cooperative  strategy  in  a  manner  that 
allows  the  UAVs  to  make  use  of  both  actual  and  predicted 
information  about  pop-up  targets  in  coordinating  their 
target  assignments.  Moreover,  since  recent  advances  in 
telecommunication  have  provided  enabling  technologies 
for  achieving  cooperative  control  of  multiple  UAVs  via  a 
communication  network  [3],  we  will  consider  networked 
UAVs  in  performing  the  PAD  operations. 

II.  System  Model 

A.  Adaptive  Markov -Chain  Model  of  Pop-up  Targets 

Pop-up  targets  are  ground  assets  that  appear  at 
unpredictable  locations  and  at  random  instants  of  time.  In 
addition,  pop-up  targets  may  stay  for  a  regular  interval  time 
and  then  disappear.  Suppose  that  the  location  of  the  next 
pop-up  target  only  depends  on  the  location  of  present 
targets  in  the  area  concerned.  Clearly,  such  discrete-time 
random  sequence  of  the  appearance  of  pop-ups  can  be 
modeled  as  a  first-order  Markov  chain  [4].  To 
mathematically  define  this  Markov-chain  model,  we  need 
to  specify  its  state  space  Q  and  one- step  transition 

matrix  P  .  First,  we  restrict  the  area  of  PAD  operation  to 
be  a  two-dimensional  rectangular  grid,  shown  in  Fig.  1, 
with  length  L  and  width  W  .  Divide  the  edge  with  length 
L  into  m  segments  and  the  edge  with  width  W  into  n 
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segments  which  will  decompose  the  original  rectangular 
region  into  mn  smaller  rectangles.  Assume  that  there 
exists  at  most  one  target  in  each  cell  at  one  time  and  no 
target  appears  right  between  cells  of  partitioned  area. 


Let  x  =  [xx,  x2]T  be  the  coordinate  of  the  pop-up  target 
with  xx  as  its  horizontal  coordinate  and  x2  as  its  vertical 
coordinate.  This  transformation  can  be  expressed  as 


Let  Zk  be  the  cell  at  which  the  pop-up  target  appears  at 
discrete-event  time  index  k  .  Note  that  the  index  k  is  the 
time  when  a  pop-up  target  shows  up.  Let  Z  =  {zk,k>  0} 
represent  a  random  sequence  of  locations  of  pop-up  targets. 
In  terms  of  our  definition  of  the  PAD  operational  area,  the 
set  of  all  the  possible  locations  that  the  pop-up  targets  can 
be  located  at,  i.e.,  the  state  space  of  Z  ,  is  composed  of 
mn  discrete  cells,  which  is  given  by  g  -  {1,2,  •  •  • ,  Nq  }  and 

Nq  =  mn  .  As  described  before,  we  model  the  predicted 

pop-up  locations  as  a  first-order  Markov-chain  such  that, 
for  any  k  >  Zk+i  is  conditionally  independent  of 
Z0,Z1,---,Zk_1  given Zk.  That  is, 

MZk+1  =  j\Zk  =  i,Zk_!  =  ik_x,“-,Z§  -  /0} 

=  Pr{Z^+ 1  =j\zk  =  /},  ijeg  ’ 

In  the  expression  above,  Zk  =  i  means  that  the  pop-up 
target  appears  in  cell  i  at  discrete  event  index  k.  We 
assume  that  one-step  transition  probabilities 
Pr{Z^+1  =  j\Zk  =/},/,  j  e  g  depend  only  on  the  states  i 
and  j  and  are  independent  of  the  index  k,  i.e., 


ce(x )  :=  int(  %2  )  xm  +  int(  X]  )  + 1,  (4) 

Win  L!  m 


where  int(-)  is  the  operator  which  takes  the  integer  part  of 
its  operand.  It  is  easy  to  verify  that  ce(x)  eg  •  On  the 
other  hand,  we  can  define  a  map  to  convert  a  given  cell 
where  a  target  is  located  to  a  two-dimensional  coordinate, 
which  is  given  by 

cr:g^[0,L]x[0,W].  (5) 

To  complete  this  definition,  we  assume  that  the  pop-up 
target  is  located  at  the  center  of  the  cell.  Thus,  if  the  target 
is  at  cell  j  e  g  ,  its  corresponding  coordinate x  =  [xx,x2]T  , 
or  v  =  cr(j ) ,  can  be  determined  as 


xx 

x2 


=  (mod(y,  /77 )  —  1 )  x 


L  |  L 
m  2m 


W 

■■  mt(— )  x  — 
n  n 


W 

+  — 
In 


if  mod(j,ra)  ^  0  ;  (6) 


where  mod(a,b)  is  the  operator  which  takes  the  value  of  the 
modulus  after  dividing  a  by  b. 

In  what  follows,  we  will  briefly  review  the  adaptive 
algorithm  to  estimate  transition  probability  (&  + 1) , 


which  is  determined  as 


Pr  {Zk+l  =j\Zk=  i]  =  ptj  for  all  i,  j  sQ.  (2) 

In  other  words,  we  are  concerned  with  homogeneous 
Markov  chains  only  in  this  paper.  Next,  we  define  two 
maps  to  simplify  some  notations  for  our  later  discussion. 
First,  we  define  a  map  to  transform  a  two-dimensional 
coordinate  into  the  corresponding  cell,  i.e., 

ce :  [0,  L]  x  [0,  W]  — »  Q  (3) 


Fig.  1  Theater  of  PAD  operations 


Pijik+l)  = 


mjjc+ 1) 

N(i,k  + 1) 


(7) 


where  N(i,k )  represent  the  number  of  pop-up  targets 
having  appeared  in  the  cell  /eg  in  the  random  sequence 
Z  by  the  discrete  event  time  k  ,  and  N(i,  j,k)  the  number 
of  transitions  from  the  current  pop-up  target  being  at  cell 
/eg  to  the  next  pop-up  target  at  cell  j  eg  in  the  random 

sequence  Z  until  k  ,  which  are  calculated  as 


N(i,j,k  + 1) 


f!V(i,M)  + 1  if  j  =  f 

[  N(i,j,k )  otherwise’ 

N(i,k  +  l)  =  N(i,k)  +  l, 


(8) 

(9) 


Essentially,  this  update  procedure  increases  the  transition 
probability  to  a  location  where  a  pop  up  has  appeared  and 
reduces  it  for  the  other  locations. 

We  need  another  procedure  to  estimate  the  uptime  at  a 
particular  location  (or  cell).  Let  tesup(i,k)  represent  the 


estimated  uptime  of  a  pop-up  target  stored  at  previously 
update  index  k  and  tup  (/,  k)  the  real  uptime  of  the  target 


staying  at  cell  i  at  previously  update  time  k  .  As  the 
targets  start  popping  up,  the  uptime  for  a  particular  pop-up 
target  is  re-calculated  when  the  target  either  is  destroyed  or 
disappears  at  next  update  time  k  + 1 ,  which  is  given  by: 
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esup 


k+l)  =  tesap(i,k)N(j,k)  +  tv(i,k) 

i,,A  +  J  N(i,k)  +  l 


(10) 


position  even  before  the  real  one  shows  up.  This  could  be 
done  if  the  predicted  information  about  pop-up  targets  is 
available. 


B.  Networked  Vehicles  Model 

Suppose  that  there  are  Nv  UAVs  and  that  the  ith  UAV  is 
modeled  as  a  Dubins’  car  model  [5]  and  flies  at  a  constant 
altitude  and  at  a  constant  velocity,  which  has  a  continuous 
time  kinematics  given  by 

Ki  =  vcos(^) 

<  xlv2  =vsin(Olv)  (11) 

~  ^maxMv 

where  xlvl  is  its  horizontal  position,  xlv2  is  its  vertical 
position,  v  is  its  transitional  (constant)  velocity,  6\  is  its 
heading  direction,  &>maxis  its  maximum  angular  velocity, 
and  - 1  <  u\  <  1  is  the  steering  input.  Rather  than  use  this 

continuous  time  representation  we  assume  that  vehicles 
will  either  travel  on  the  minimum  turn  radius  or  on  straight 
lines.  It  is  then  possible  to  analytically  write  down  the 
formulas  for  the  vehicle  trajectories  (e.g.  in  terms  of  arc 
segments  on  circles  and  line  segments  [5]).  All  UAVs  are 
connected  via  communication  network  [6]  and  assumed  to 
have  perfect  knowledge  of  threats  that  have  been  popped 
up. 

C.  Multi-UAV Simulation  Test  bed 

In  order  to  implement  and  evaluate  cooperative  control 
strategy,  we  use  a  Matlab  simulation  first  developed  at 
AFRL  [7]  and  then  extended  by  the  authors  as  a  test  bed. 
In  this  paper,  we  consider  the  area  of  PAD  operation  in 
Fig.l  with  length  of  L  -  5000  meters  and  width  of 
W  =  5000  meters,  and  let  m  =  n  =  10  so  that  the  entire  area 
is  divided  into  100  cells.  Here  we  assume  that,  for  each 
current  pop-up  location,  the  maximum  number  of  next 
possible  pop-up  locations  is  five  and  each  of  the  next  pop¬ 
up  locations  is  not  identical  to  the  current  pop-up  location. 
We  also  assume  that  the  first-order  Markov  chain  model 
and  the  estimated  uptime  of  pop-up  targets  at  each  cell  have 
been  obtained  using  the  adaptive  approach  before  UAVs 
start  their  PAD  mission. 

III.  Maneuvering  Uavs  To  Pop-up  Threats 

To  aid  in  the  development  of  cooperative  control 
strategies  among  UAVs,  first  we  need  to  address  the  issue 
of  how  to  maneuver  UAV  to  pop-up  targets.  In  the  absence 
of  knowing  the  information  about  predicted  pop-up  threats, 
a  UAV  often  waits  for  the  next  actual  pop-up  target  to 
appear  and  then  maneuvers  to  that  target.  Such  strategy  is 
often  referred  to  as  a  simplistic  and  inactive  strategy.  We 
call  this  type  of  strategy  “WA”  strategy.  On  the  other 
hand,  a  UAV  may  decide  to  move  to  an  intermediate 


A.  Strategies  for  locating  UAVs  intermediate  destination 

Consider  a  present  pop-up  target  at  cell  i ,  and  the 
corresponding  next  pop-up  targets  predicted  from  the 
Markov-chain  model  as  illustrated  in  Fig.  2.  Let  S  denote 
the  set  of  cells  where  those  predicted  pop-up  targets  are 
located,  which  is  S  =  j  j) ,  j? ,  •  •  • ,  j- } ,  where  p..i  >  0  for  all 

/  =  1,  •  •  • ,  s  .  Moreover,  let  xl  -  [x[ ,  xl2  ]T  be  the  coordinate 
of  the  /^predicted  pop-up  targets  and xl  =  cr(j\ ) , 
for l  -  1,2 ,•••,£  . 

Definition  1.  A  predicted  location  xML  =  [x^L ,  x2L  ]T 
selected  as 

xML  =  xl*  with  /*  =  argmax{/u./ } 

is  called  maximum  likelihood  (ML)  locating  strategy. 
Definition  2.  A  predicted  location xBW  =  [xBW  ,  x2w  ]T 
given  by 


5 


i=i 


is  called  a  Bayesian  weighted  (BW)  locating  strategy. 
Definition  3.  A  predicted  location  xuw  =  [x±w  ,  x2w  ]T 
given  by 


is  called  a  uniformly  weighted  (UW)  locating  strategy. 

We  define  completion  time  as  the  time  that  it  takes  a  UAV 
to  reach  the  next  pop-up  target  after  it  becomes  visible. 
Clearly,  the  completion  time  depends  on  when  this  pop-up 
target  shows  up  and  which  predicted  target  in  the  set  S  will 
be  appeared  as  a  real  target.  Clearly,  the  completion  time  is 
a  random  variable.  Hence,  we  will  use  the  expected  value 
of  completion  time  as  a  performance  index  to  evaluate 
various  locating  strategies  developed  so  far. 

Let  tWA  be  the  completion  time  when  using  WA  strategy, 
and  let  E(tWA )  denote  the  expected  value  of  tWA ,  which 
can  be  calculated  as 

s 

E ( *WA  )  =  X  pij !  d (i'  1 1 2) 

1=1 

where  d(i,j )  is  the  minimum  flying  distance  for  UAV 
from  cell  i  to  cell  j  .  Similarly,  we  define  tML ,  tBW  and 
tuw  as  the  completion  times  using  ML  locating  strategy, 
BW  locating  strategy  and  UW  locating  strategy, 
respectively. 
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Fig.  2.  Present  target  at  cell  l  and  the  corresponding  predicted  pop-up 
targets  in  the  set  S. 

The  expected  values  of  these  completion  times  can  be 
computed  as 

s 

E(tML )  =  V  p..i  d(jj* ,  j\ )  /  v  where  l*  =  arg  ma x{p  , }  (13) 

v  i  1  lJ  i 

1=1 , 

1*1* 

E(hw )  =  X  pij !  d^BW  ’  j‘ )  /  v  where  jBW  =  ceVBW )  (14) 
1=1 

E(tuw )  =  ^  p..,d(juw ,  j\ )  /  v  where  juw  =  ce(xuw)  (15) 
1=1 

With  the  expressions  (12)-(15),  we  can  easily  figure  out 
under  what  conditions  the  performances  of  three  locating 
strategies  are  better  than  the  WA  strategy.  Obviously,  if 

P€d{i,  j‘*)  +  £  p..,  (d(i,  j\ )  -d(f,  j\  ))>  0 .  (16) 

1=1, 

1*1* 

Similarly,  the  condition 

^  ( d (/,  j\ )  -  d(  j* ,  j\ ) ) >  0  where  jcy-*  =  xBW  or  xuw  (17) 
1=1 

Ensures  that  the  performance  of  the  weighted  locating 
strategies  is  better  than  WA  strategy. 

We  can  also  obtain  other  conditions  for  evaluating  the 
performance  of  the  ML  locating  strategy  and  the  weighted 
locating  strategies  in  a  similar  way.  In  general,  the  hold  of 
those  conditions  will  depend  upon  either  the  transition 
probabilities  or  spatial  distribution  of  the  predicted  targets, 
or  both  of  them. 

B.  Monte  Carlo  Evaluations 

In  this  section,  we  would  like  to  perform  the  Monte  Carlo 
simulations  to  evaluate  various  locating  strategies.  It  is 
clear  that  transition  probabilities  and  spatial  distribution  of 
the  predicted  targets  are  two  extremely  important  factors  in 
determining  which  strategy  works  best. 

Experiment  3.1  In  this  experiment,  we  consider  the 
effects  of  transition  probabilities  on  the  performance  of 
various  strategies.  First,  we  define 

S'  =  max  {»../}-  min  {»../}  where  p..t  >0 

pr  i  ij  i  i  Ui 

as  the  variation  of  the  transition  probabilities  with  respect 
to  the  present  target  at  cell  i .  For  example,  if  Slpr  =  0  ,  this 


means  p..i  -p..v>  0  for  /  ^  /' .  We  choose  a  scenario 

with  100  sets  of  pop-up  locations  and  perform  the  10,000 
Monte  Carlo  runs  for  each  set  letting  the  variation  of  the 
transition  probabilities  vary  from  1%  to  35%.  The  results 
are  shown  in  Fig.  3. 

We  observe  that  the  performances  of  three  locating 
strategies  (ML,  BW  and  UW)  are  significantly  better  than 
the  WA  strategy  for  each  probability  distribution.  Since 
the  predicted  information  is  not  utilized  in  the  WA  strategy, 
the  performance  of  this  strategy  does  not  vary  much  as  we 
vary  the  transition  probability  distribution.  When  the 
variation  of  the  transition  probabilities  is  small,  both  UW 
and  BW  locating  strategies  work  better  than  ML  locating 
strategy.  As  the  variation  8lpr  increases,  the  averaged 

completion  time  using  the  ML  locating  strategy  falls  off 
constantly  and  results  in  much  better  performance  than 
weighted  locating  strategies.  The  error  bars  show  the 
standard  deviation  of  every  data  point  which  corresponds  to 
the  average  completion  time  of  100  independent  runs. 

Experiment  3.2  In  this  experiment,  we  examine  the 
effects  of  the  other  factor,  i.e.,  the  spatial  distribution  of 
predicted  targets,  on  the  performance  of  various  strategies. 
We  fix  the  location  of  the  present  pop-up  target  at  cell  1 
and  take  five  sets  of  spatial  distributions  of  predicted 
targets  as  shown  in  Fig.  4. 

For  each  configuration  of  distribution,  we  assume  that 
the  predicted  pop-up  targets  are  uniformly  distributed  in  the 
area  composed  of  the  corresponding  set  of  cells.  Hence, 
the  predicted  pop-up  targets  are  spread  out  as  the  set 
number  increases.  Fig.  5  compares  the  outcome  of 
simulations  with  5000  sets  of  pop-up  targets  for  each 
distribution  and  1000  Monte  Carlo  runs  for  each  set. 

We  observe  that  three  locating  strategies  present  better 
performance  than  that  using  WA  strategy.  As  the  targets 
are  more  spread  out,  we  observe  that  the  difference 
between  two  weighted  locating  strategies  becomes  larger. 
In  particular,  the  ML  locating  strategy  works  best  in  this 
simulation.  However,  the  standard  deviation  of  the 
performance  of  the  ML  locating  strategy  is  worse  than 
those  two  weighted  locating  strategies.  This  can  be  seen  in 
Fig  6. 

In  most  cases,  the  ML  locating  strategy  works  best  in 
terms  of  minimizing  the  expected  time  to  reach  the  next 
pop-up  target  after  it  appears. 

IV.  Cooperative  Control  Strategies  of  Networked 
UAVs  for  Pad  Mission 

Basically,  it  is  desirable  to  have  more  than  one  UAV 
performing  persistent  area  denial  since  multiple  UAVs  can 
cooperate  by  sharing  relevant  information  and  try  to 
accomplish  the  task  more  efficiently.  In  this  section,  we 
will  define  specific  version  of  our  cooperative  control 
strategy  by  involving  the  information  of  predicted  pop-up 
targets  in  our  development. 
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A.  Cooperative  Strategy  Using  Mixed  Information 

1 )  Initial  Cooperative  Target  Assignment :  First,  let  us 
consider  the  allocation  of  multiple  UAVs  to  the  targets  that 
are  currently  observed.  This  can  be  done  by  using  integer 
programming  (IP)  approach  [8].  The  cost  function  and 
constraints  are  formulate 

U{k)nx(k)  1 

min  \  T  V  d(x'v , xJr  )Q (i,  j)\  (18) 

J 

subject  to  the  constraints 

nx(k) 

XQ0^)  =  1  (19) 

7=1 

nx(k) 

XQ0'>7)  =  1  (20) 


where  Q(z,  j)  is  the  assignment  function  which  is  given  by 


mj) 


1  if  UAV  i  is  assigned  to  target  j 
[0  otherwise 


(21) 


The  constraints  (19)  and  (20)  means  that  each  UAY  can  be 
assigned  to  a  single  target  only  and  vise  versa.  For  the 
purpose  of  illustration,  we  currently  consider  two  UAYs, 
i.e.,  Nv  =  2 .  This  algorithm  actually  assigns  each  UAY  to 
the  target  closest  to  its  current  location  in  the  area.  Let  AV 
{UV)  and  AT  {UT)  denote  the  set  of  assigned  (unassigned) 
UAVs  and  assigned  (unassigned)  pop-up  targets, 
respectively.  In  this  algorithm,  a  UAV  (target)  assigned  to 
a  dummy  target  (UAY)  is  called  an  unassigned  UAV 
(target).  2)  Cooperative  Targeting  Adjustment:  It  is 
possible  that  in  certain  situation,  a  single  UAV  is  sufficient 
to  destroy  two  or  more  targets  close  to  each  other.  In 
addition,  the  allocation  of  UAYs  to  targets  depends  on  the 
remaining  time  of  the  appearance  of  targets  also,  i.e. 
{tesUp(i)J  e  Q) .  Consider  the  situation  where 


Dim(AT)  =  2  and  AT  =  {jx,  j2 } .  This  means  all  the  UAVs 
are  assigned  to  attack  the  two  targets  because  Nv  =  2 .  We 
claim  that  two  targets  are  close  to  each  other  if  the  distance 
between  them  is  less  than  a  threshold  dth  .  In  this  case,  one 

UAV,  say  UAY  l  ,  can  be  assigned  to  attack  both  of  them 
at  the  same  time.  If  the  set  UTis  not  empty,  then  the  other 

•  t  •* 

UAV  l  ^  l  will  be  assigned  to  a  target  in  UT  closest  to  it. 
If  Dim(UT)  >  2  ,  UAV  l  may  determine  how  many  targets 
it  can  deal  with  using  the  adjustment  algorithm.  If 
UT  =  0 ,  the  sets  AV  and  UV  are  updated  as 
:  AV  =  AV  /{ UAV }  and  UV  =  UV  U  {UAVv } . 


Fig.  3.  Completion  time  for  various  variations  of  transition  probabilities 


Distance  in  km 

Fig.  4.  Five  sets  of  spatial  distributions 


Fig.  5.  Completion  time  for  various  spatial  distribution  of  the  predicted 
pop-up  targets 
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Fig.  6.  Standard  deviations  for  three  locating  strategies 
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3 )  Predicted  Target  Assignment:  In  the  case  that  UT  =  0 
but  UV  ^  0 ,  the  UAVs  in  set  UV  will  determine  an 
intermediate  location  to  move  by  using  the  locating 
strategies  developed  in  step  2)  based  on  the  predicted 
information  obtained  from  Markov  model,  instead  of 
waiting  for  the  next  pop-up  target  to  appear. 

B.  Illustrative  Examples 

The  cooperative  control  strategies  developed  in  the 
previous  section  are  tested  using  the  simulation  performed 
from  the  AFRL  software.  The  simulations  here  will  have  a 
group  of  two  UAVs  assigned  for  four  pop-up  targets  in 
total  in  the  given  area  and  the  initial  situation  is  shown  in 
Fig.  7.  Note  that  the  present  targets  1  and  2  are  close  to 
each  other.  As  a  result,  UAV  1  will  be  assigned  to  reach 
both  of  them.  Hence,  UV=  { UAV  2 } . 

In  the  first  simulation,  we  test  the  performance  of  the 
cooperative  strategy  without  using  the  predicted 
information  (i.e.,  WA  strategy).  By  viewing  a  movie  of 
Fig.  7,  we  see  that  the  unassigned  UAV  2  is  just  hovering 
around  its  initial  location  to  wait  for  the  appearance  of  the 
next  pop-up  target,  i.e.,  target  3.  As  a  result,  when  pop-up 
target  3  appears,  UAV  2  starts  moving  to  it  and  finally,  the 
UAV  2  misses  its  target.  Similarly,  UAV  1  fails  to  reach 
the  pop-up  target  4  in  time.  In  the  end,  we  notice  that  the 
team  of  UAVs  missed  two  pop-up  targets  in  total  and  it 
took  them  1501  seconds  for  the  overall  simulation. 

We  then  performed  the  same  simulation  except  that  the 
unassigned  UAV  now  decides  to  move  to  an  intermediate 
location  using  the  ML  locating  strategy.  In  this  simulation, 
we  see  that  the  UAV  2  flies  to  the  destination  where  the 
predicted  pop-up  target  with  the  highest  transition 
probability  is  located.  When  the  pop-up  target  3  shows  up, 
UAV  2  is  much  closer  to  it  and  thus  continues  to  move  to 
it.  As  a  result,  UAV  2  catches  the  target  3  successfully. 
We  also  observe  that  UAV  1,  after  accomplishing  the  pre¬ 
assignment,  then  heads  towards  the  predicted  location  using 
the  ML  locating  strategy,  and  reach  the  target  4  in  a  timely 
manner.  In  the  end,  all  the  four  pop-up  targets  are  reached 
successfully  by  taking  advantage  of  the  predicted 
information  and  the  overall  mission  takes  the  group  of 
UAVs  1401  seconds,  a  shorter  simulation  time,  to  complete 
the  PAD  task. 

V.  Concluding  Remarks 

This  research  addresses  the  coordination  of  the  multiple 
UAVs  in  PAD  operations  for  pop-up  threats.  We  present 
various  locating  strategies  to  move  a  UAV  to  an 
intermediate  position  using  predicted  information.  Monte 
Carlo  runs  show  that  the  ML  locating  strategy  has  an 
average  performance  superior  to  other  locating  strategies  in 
many  cases.  Cooperative  strategies  considered  in  this  paper 
are  designed  to  achieve  the  objectives  of  UAVs  based  on 


Fig.  7  Initial  Situation 

available  information  about  predicted  pop-up  targets,  thus 
enabling  better  results  for  their  PAD  operations.  Our 
simulation  results  have  demonstrated  that  the  developed 
cooperative  strategies  using  mixed  information  about  pop¬ 
up  targets  in  an  appropriate  way  can  reduce  mission  time 
and  increase  the  number  of  the  targets  reached,  and  thus 
improve  the  overall  performance  of  PAD  operation. 
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